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Ergodic Convergence Rates of 
Markov Processes — Eigenvalues, 
Inequalities and Ergodic Theory 



Mu-Fa Chen* 
Abstract 

This paper consists of four parts. In the first part, we explain what 
eigenvalues we are interested in and show the difficulties of the study on the 
first (non-trivial) eigenvalue through examples. In the second part, we present 
some (dual) variational formulas and explicit bounds for the first eigenvalue 
of Laplacian on Riemannian manifolds or Jacobi matrices (Markov chains). 
Here, a probabilistic approach — the coupling methods is adopted. In the third 
part, we introduce recent lower bounds of several basic inequalities; these are 
based on a generalization of Cheeger's approach which comes from Riemannian 
geometry. In the leist part, a diagram of nine different types of ergodicity and 
a table of explicit criteria for them are presented. These criteria are motivated 
by the weighted Hardy inequality which comes from Harmonic analysis. 
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I. Introduction 

We will start by explaining what eigenvalues we are interested in. 

1.1 Definition. Consider a birth-death process with a state space £' = {0, 1, 2, • • • , 
n} (n ^ oo) and an intensity matrix Q = (qij): qk,k-i = > (1 ^ fc 5j n), 
qkM+i = fefc > (0 < fc ^ n - 1), qk,k = -{ak + bk), and = for other i ^ j. 

Since the sum of each row equals 0, we have Ql = = 0-1. This means that 
the Q- matrix has an eigenvalue with an eigenvector 1. Next, consider the finite 
case of n < oo. Then, the eigenvalues of —Q are discrete: = Ao < Ai ^ • • • ^ A„. 
We are interested in the first (non-trivial) eigenvalue Ai = Ai — Aq (also called 
spectral gap of Q). In the infinite case (n = oo), Ai can be 0. Certainly, one can 
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consider a self-adjoint elliptic operator in R'^, the Laplacian A on manifolds, or an 
infinite-dimensional operator as in the study of interacting particle systems. 

1.2 DifHculties. To get a concrete feeling about the difficulties of this topic, let 
us first look at the following examples with a finite state space. When E = {0, 1}, 
it is trivial that Ai = oi + 6o- The result is nice because when either ai or bo 
increases, so does Ai. When E = {0, 1, 2}, we have four parameters bo, 6i, oi, 02 and 
Ai = 2-1 [ai + a2 + bo + bi- ^ {ai - a2 + bo - bi)^ + 'iaibi] . When E = {0, 1, 2, 3}, 
we have six parameters: 60, 61, 62, ai, ^2, as- In this case, the expression for Ai is 
too lengthy to write. The roles of the parameters are inter-related in a complicated 
manner. Clearly, it is impossible to compute Ai explicitly when the size of the 
matrix is greater than five. 

Next, consider the infinite state space E = {0,1,2,- ■ ■}. Denote the eigenfunc- 
tion of Ai by g and the degree of g by D{g) when g is polynomial. Three examples 
of the perturbation of Ai and D(g) are listed in Table l.f. 



b^{^ ^ 0) 


ai{i ^ 1) 


Ai 


D{g) 


i + c{c> 0) 


2i 


1 


1 


i + 1 


2i + 3 


2 


2 


i + 1 


2i+{4 + V2) 


3 


3 



Table 1.1 Three examples of the perturbation of Ai and D{g) 

The first line is the well known linear model for which Ai = 1, independent of the 
constant c > 0, and g is linear. Keeping the same birth rate, bi = i + 1, changes 

the death rate from 2i to 2z -I- 3 (resp. 2i + 4 -I- \/2), which leads to the change of 
Ai from one to two (resp. three). More surprisingly, the eigenfunction g is changed 
from linear to quadratic (resp. triple). For the other values of Oi between 2i, 2i + 3 
and 2i + 4: + \f2, \\ is imknown since g is non-polynomial. As seen from these 
examples, the first eigenvalue is very sensitive. Hence, in general, it is very hard to 
estimate Ai. 

In the next section, we find that this topic is studied extensively in Riemannian 
geometry. 

n. New variational formula for the first eigenvalue 

2.1 story of estimating Ai in geometry. At first, we recall the study of Ai in 
geometry. 

Consider Laplacian A on a compact Riemannian manifold {M,g), where g 
is the Riemannian metric. The spectrum of A is discrete: • • • < — A2 < — Ai < 
— Ao = (may be repeated). Estimating these eigenvalues Afc (especially Ai) is very 
important in modern geometry. As far as we know, five books, excluding those 
books on general spectral theory, have been devoted to this topic: Chavel (1984), 
Berard (1986), Schoen and Yau (1988), Li (1993) and Ma (1993). For a manifold 
M, denote its dimension, diameter and the lower bound of Ricci curvature by d, D, 
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and K (RicciM ^ Kg), respectively. We are interested in estimating Ai in terms of 
these three geometric quantities. It is relatively easy to obtain an upper bound by 
applying a test function / e (M) to the classical variational formula: 



Ai = inf I ^ l|V/||'dar : / e C\M), j fdx = 0, J fdx = l|, (2 



0) 



where "dx" is the Riemannian volume element. To obtain the lower bound, however, 
is much harder. In Table 2.1, we list eight of the strongest lower bounds that have 
been derived in the past, using various sophisticated methods. 



Author(s) 


Lower bound 


A. Lichnerowicz (1958) 




(2.1) 


P. H. Berard, G. Bcsson 
& S. Gallot (1985) 


.rr^cos^- 


itdtl'/'i 
itdt J 


= d- 1 > 0. (2.2) 


P. Li & S. T. Yau (1980) 




(2.3) 


J. Q. Zhong & 
H. C. Yang (1984) 


^' K,0. 




(2.4) 


P. Li & S. T. Yau (1980) 




1 


-, K ^0. (2.5) 


£>2(d_ i)exp 


1 + Vl + 16a2 


K. R. Cai (1991) 


— +K, K<0. 




(2.6) 


H. C. Yang (1989) & 
F. Jia (1991) 


^e-«, if d > 5, K^O. 


(2.7) 


H. C. Yang (1989) & 
F. Jia (1991) 


e-"' if 
2D^ ' 


2 ^ d ^ 4, K ^0, (2.8) 



Table 2.1 Eight lower bounds of Ai 
In Table 2.1, the two parameters a and a' arc defined as a = Di/li^Kd- l)/2 and 
a' = Dy/\K\{{d-l)W2)/2. Among these estimates, five ((2.1), (2.2), (2.4), (2.6) 
and (2.7)) are sharp. The first two are sharp for the unit sphere in two or higher 
dimensions but fail for the unit circle; the fourth, the sixth, and the seventh are all 
sharp for the unit circle. As seen from this table, the picture is now very complete, 
due to the efforts of many geometers in the past 40 years. Our original starting 
point is to learn from the geometers and to study their methods, especially the 
recent new developments. In the next section, we will show that one can go in the 
opposite direction, i.e., studying the first eigenvalue by using probabilistic methods. 
Exceeding our expectations, we find a general formula for the lower bound. 

2.2 New variational formula. Before stating our new variational formula, we 
introduce two notations: 



C(r) = cosh''- 



-K 
d-1 



r G (0, D). J- = {/ G C[0, : / > on (0, £>)}. 
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Here, we have used all the three quantities: the dimension d, the diameter D, and 

the lower bound K of Ricci curvature. 



Theorem 2.1[General formula] (Chen & Wang (1997a)). 

4/(r) 



Ai > sup inf 



/e>re(o,D) C(s)-ids C{u)f{u)du 



=:6 



(2.9) 



The new variational formula has its essential value in estimating the lower 

bound. It is a dual of the classical variational formula in the sense that "inf" in 
(2.0) is replaced by "sup" in (2.9). The classical formula can be traced to Lord S. 
J. W. Rayleigh (1877) and E.Fischer (1905). Noticing that these two formulas (2.0) 
and (2.9) look very different, which explains that why such a formula (2.9) has never 
appeared before. This formula can produce many new lower bounds. For instance, 
the one corresponding to the trivial function / = 1 is non-trivial in geometry. 
Applying the general formula to the test functions sin(Q;r) and cosh''""^ (ar) sin(/3r) 
with a = Dy'lKKd- l)/2 and /3 = 7r/(2D), we obtain the following: 

Corollary 2.2 (Chen& Wang (1997a)). 



Ai ^ 



dK 
d-1 



-cos'^ 


■Dl K ■ 


2\d- 
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2D^K 


cosh^-'' 


D 


7r4 


"2" 



-K 



d- 1 



d>l, K^O, (2.10) 
d>l, K ^0. (2.11) 



Applying this formula to some very complicated test functions, we can prove 

the following result: 

Corollary 2.3 (Chen, Scacciatelli and Yao (2002)). 



Ai > TT^/D'^ + K/2, K e 



(2.12) 



The corollaries improve all the estimates (2.1) — (2.8). Especially, (2.10) im- 
proves (2.1) and (2.2), (2.11) improves (2.7) and (2.8), and (2.12) improves (2.3) 
and (2.6). Moreover, the linear approximation in (2.12) is optimal in the sense that 
the coefficient 1/2 of is exact. 

A test function is indeed a mimic of the eigenfunction, so it should be chosen 
appropriately in order to obtain good estimates. A question arises naturally: does 
there exist a single representative test function such that we can avoid the task 
of choosing a different test function each time? The answer is seemingly negative 
since we have already seen that the eigenvalue and the eigenfunction are both very 
sensitive. Surprisingly, the answer is affirmative. The representative test function, 
though very tricky to find, has a rather simple form: /(r) = C{s)~^ds. This 
is motivated from the study of the weighted Hardy inequality, a powerful tool in 
harmonic analysis (cf. Muckenhoupt (1972), Opic and Kufner (1990)). 
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Corollary 2.4 (Chen (2000)). For the lower bound of Xi given in Theorem 2.1, 
we have 



4^"^ > 6 > where (2.13) 

5= sup ([ C(s)-MsV / C(s)dsY C(s) =cosh'^ 
re(o,r>) \J0 J \Jr J 



s / -K 



2 \ d-1 



Theorem 2.1 and its corollaries are also valid for manifolds with a convex 
boundary endowed with the Neumann boundary condition. In this case, the esti- 
mates (2.1) — (2.8) are conjectured by the geometers to be correct. However, only 
the Lichnerowicz's estimate (2.1) was proven by J. F. Escobar in 1990. The others 
in (2.2) — (2.8) and furthermore in (2.10) — (2.13) are all new in geometry. 

On the one hand, the proof of this theorem is quite straightforward, based 
on the coupling introduced by Kendall (1986) and Cranston (1991). On the other 
hand, the derivation of this general formula requires much effort. The key point is 
to find a way to mimic the eigenfunctions. For more details, refer to Chen (1997). 

Applying similar proof techniques to general Markov processes, we also obtain 
variational formulas for non-compact manifolds, elliptic operators in M"^ (Chen and 
Wang (1997b)), and Markov chains (Chen (1996)). It is more difficult to derive the 
variational formulas for the elliptic operators and Markov chains due to the presence 
of infinite parameters in these cases. In contrast, there are only three parameters 
(rf , D, and K) in the geometric case. In fact, formula (2.9) is a particular example 
of our general formula (which is complete in dimensional one) for elliptic operators. 

To conclude this part, we return to the matrix case introduced at the beginning 
of the paper. 

2.3 Birth-death processes. Let bi > 0{i > 0) and > 0{i ^ 1) be the birth and 
death rates, respectively. Define /xq = 1, /^i = 6o • • • ■ ■ ■ai{i ^ 1). Assume 

that the process is non-explosive: 

Z!feLo(^feA*fe)~^ Y.i=o Mi = 00 and moreover ij, = Y,^^i^ < oo- (2-14) 
The corresponding Dirichlet form is D{f) = Y,, T^ih{fi+i- fif , D{D) = {f €L'^{-k) : 
D{f) < oo}. Here and in what follows, only the diagonal elements D{f) are written, 
but the non-diagonal elements can be computed from the diagonal ones by using 
the quadrilateral role. We then have the classical formula Ai = {D{f) : 7r(/) = 
0,7r(/^) = l}. Define y = {/ : /o = 0, there exists k: 1 ^k^oo so that /; = fi/^,k 
and / is strictly increasing in [0, A:]}, J'" = {/ : /o = 0, / is strictly increasing}, 
and Ii{f) = [iJ.ibi{fi+i - /,)]~i Y.j^i+i l^jfr f = f - 7i"(/)- Then we have the 
following results: 

Theorem 2.5 (Chen (1996, 2000, 2001))^ Under (2.14), we have 

(1) Dual variational formula, inf sup/j(/)~^ = Ai = sup inf/j(/)~^. 



^Due to the limitation of the space, the most of the author's papers during 1993-2001 are 
not Usted in References, the readers are urged to refer to [11]. 
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(2) Explicit estimate. ii5~^ ^ Ai ^ (4^)~^, where 6 = sup ^ {f^jbj)~^ Yl l^j- 

(3) Approximation procedure. There exist explicit sequences r]'^ and ri" such that 

Here the word "dual" means that the upper and lower bounds are interchange- 
able if one exchanges "sup" and "inf" . With slight modifications, this result is also 
valid for finite matrices, refer to Chen (1999). 

HI. Basic inequalities and new forms of Cheeger's 
constants 

3.1 Basic inequalities. We now go to a more general setup. Let tt) be a 

probability space satisfying {{x,x) : x ^ E} ^ £ x £. Denote by L'p{'k) the usual 
real L^-space with norm || • ||p. Write || • || = || • ||2- 

For a given Dirichlet form {D,T>{D)), the classical variational formula for the 
first eigenvalue Ai can be rewritten in the form of (3.1) below with an optimal 
constant C = X^^ . From this point of view, it is natural to study other inequalities. 
Two additional basic inequalities appear in (3.2) and (3.3) below. 

Poincare inequality : Var(/) < CD{f), f e L'^{tt), (3.1) 
Logarithmic Sobolev inequality: J f'^\ogj^j^dn^CD{f), fGL'^{n), (3.2) 
Nash inequality : Var(/) ^ CDify/PyWl^" , f e L^^tt), (3.3) 

where Var(/) = Tr{p) - Tr{ff, 7r(/) = / fdw, p G (1, oo) and 1/p + 1/q = 1. The 
last two inequalities are due to Gross (1976) and Nath (1958), respectively. 

Our main object is a symmetric (not necessarily Dirichlet) form {D, V{D)) on 
L^(7r), corresponding to an integral operator (or symmetric kernel) on {E,£): 

D{f) = \l J(dx,dy)[/(^/)-/(x)]^ V{D) = {f &L\'K):D{f)<<x^}, (3.4) 

where J is a non-negative, symmetric measure having no charge on the diagonal 
set {{x^x) : X £ E}. A. typical example is the reversible jump process with a g-pair 
{q{x), q{x,dy)) and a reversible measure tt. Then J(d.x,dy) = Tr{dx)q{x,dy). 

For the remainder of this part, we restrict our discussions to the symmetric 
form of (3.4). 

3.2 Status of the research. An important topic in this research area is to study 
under what conditions on the symmetric measure J do the above inequalities hold. 
In contrast with the probabilistic method used in Part (E), here we adopt a ge- 
neralization of Cheeger's method (1970), which comes from Riemannian geometry. 
Naturally, we define Ai := inf {£)(/) : 7r(/) = 0, ||/|| = 1}. For bounded jump 
processes, the fundamental known result is the following: 
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Theorem 3.1(Lawler & Sokal (1988)). Ai > — , where 

f. nidxjqix, A") 

k= inf , — -- — and M = supg(x). 

7r(A)e(o,i) 7r(A)A7r(A^) 

In the past years, the theorem has been cohectcd into six books: Chen (1992), 

Sinclair (1993), Chung (1997), Saloff-Coste (1997), Colin de Verdiere (1998), Al- 

dous, D. G. & Fill, J. A. (1994-). Prom the titles of the books, one can see a wide 

range of the apphcations. However, this result fails for the unbounded operator. 

Thus, it has been a challenging open problem in the past ten years to handle the 

unbounded case. 

As for the logarithmic Sobolev inequality, there have been a large number of 
publications in the past twenty years for differential operators. (For a survey, see 
Bakry (1992) or Gross (1993)). Still, there are very limited results for integral 
operators. 

3.3 New results. Since the symmetric measure can be unbounded, we choose a 
symmetric, non-negative function r{x,y) such that 

^r^w, ,x . J(da;,dw) , jW(dx,E) , 

J("){dx, dy) := I{r{a:,y)<'>o} ^^^^^^^ > 0) Satisfies ^^^^^ < 1, 7r-a.s. 

For convenience, we use the convention = J. Corresponding to the three 
inequalities above, we introduce the following new forms of Cheeger's constants. 



Constant fc^"^ 



Inequality 



inf ^^—^ —4 (Chen & Wang(1998)) 



Poincare 



lim inf , ^ ' = (Wang (2001a)) 

'•^0 7r(A)e(o,r] 7r(A)Vlog[e + 7r(A)-i] 

- J^-ni X A^TTMI) (Chen (2000)) 
S^oo ■k{A)>o Tr{A)y/l - log 7r(A) 



Log. Sobolev 



Log. Sobolev 



inf ,,,, (Chen (1999)) 

7r(A)e(0,l) [7r(A) A7r(A°)](2g-3)/(29-2) ^ ^ 



Nash 



Table 3.1 New forms of Cheeger's constants 

Our main result can be easily stated as follows. 

Theorem 3.2. kM^^^ > the corresponding inequality holds. 

In other words, we use J^^/^' and J^^' to handle the unbounded J. The first 
two kernels come from the use of Schwarz inequality. This result is proven in four 
papers quoted in Table (3.1). In these papers, some estimates which are sharp or 
qualitatively sharp for the upper or lower bounds are also presented. 

IV. New picture of ergodic theory and explicit 
criteria 

4.1 Importance of the inequalities. Let (Pt)t^o be the semigroup determined 
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by a Dirichlct form {D,'D{D)). Then, various apphcations of the inequaUties are 

based on the following results: 

Theorem 4.1 (Liggett (1989), Gross (1976) and Chen (1999)). 

(1) Poincare inequality \\Ptf - 7T{f)f = Var(P(/) Var(/) exp[-2Aii]. 

(2) Logarithmic Sobolev inequality exponential convergence in entropy: 
Ent{Pt.f) ^ Ent(/)exp[-2at], where Ent(/) = 7r(/log/) -7r(/) log ||/||i. 

(3) Nash inequality ^ Var(Pt/) ^ C\\f\\i/t^-i. 

In the context of diffusions, one can replace "=>" by "^s=>" in part (2). There- 
fore, the above inequalities describe some type of L^-crgodicity for the semigroup 
(-Pt)t^o- These inequalities have become powerful tools in the study on infinite- 
dimensional mathematics (phase transitions, for instance) and the effectiveness of 
random algorithms. 

4.2 Three traditional types of ergodicity. The following three types of ergo- 
dicity are well known for Markov processes. 

Ordinary ergodicity : lim ||pt(a;, •) — 7r||var = 

t — *oo 

Exponential ergodicity : \\pl(x, •) — 7r||var ^ C(x)e~°'* for some a > 
Strong ergodicity : lim sup \ \pt{x, ■) — 7r||var = 

t — ^oo ^ 

lim e^* sup ||}3t(a;, •) — 7r||var = for some /3 > 

t — ^oo a; 

where pt{x,dy) is the transition function of the Markov process and || • ||var is the 
total variation norm. They obey the following implications: 

Strong ergodicity => Exponential ergodicity => Ordinary ergodicity. 

It is natural to ask the following question, does there exist any relation between 
the above inequalities and the three traditional types of ergodicity? 

4.3 New picture of ergodic theory. 

Theorem 4.2 (Chen (1999), ...). For reversible Markov processes with densities, 
we have the diagram shown in Figure 4-i- 

Nash inequality 

// ^ 
Logarithmic Sobolev inequality L^ -exponential convergence 

^ t 
Exponential convergence in entropy Strong ergodicity 

ii- ii- 

Poincare inequality <^=> Exponential ergodicity 

L^ -algebraic ergodicity 

Ordinary ergodicity 
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Figure 4.1 Diagram of nine types of ergodicity 

In Figure 4-1, LF' -algebraic ergodieity means that Var(Pt/) ^ CV{f)t^~'^ (t > 0) 
holds for some V having the properties (cf. Liggett (1991)): V is homogeneous of 
degree two (in the sense that V{cf + d) = c'^V{f) for any constants c and d) and 
V{f) < oo for all functions f with finite support. 

The diagram is complete in the following sense: each single-side implication can 
not be replaced by double-sides one. Moreover, strong ergodicity and logarithmic 
Sobolev inequality (resp. exponential convergence in entropy) are not comparable. 
With exception of the equivalences, all the implications in the diagram are suitable 
for more general Markov processes. Clearly, the diagram extends the ergodic theory 
of Markov processes. 

The diagram was presented in Chen (1999), originally for Markov chains only. 
Recently, the equivalence of L-'^-exponcntial convergence and strong ergodicity was 
mainly proven by Y. H. Mao. A counter-example of diffusion was constructed by 
Wang (2001b) to show that strong ergodicity does not imply exponential conver- 
gence in entropy. For other references and a detailed proof of the diagram, refer to 
Chen (1999). 

4.4 Explicit criteria for several types of ergodicity. As an application of 
the diagram in Figure 4.1, we obtain a criterion for the exponential ergodicity of 
birth-death processes, as listed in Table 4.2. To achieve this, we use the equivalence 

of exponential ergodicity and Poincarc inequality, as well as the explicit criterion for 
Poincare inequality given in part (3) of Theorem 2.5. This solves a long standing 
open problem in the study of Markov chains (cf. Anderson (1991), §6.6 and Chen 
(1992), §4.4). 

Next, it is natural to look for some criteria for other types of ergodicity. To 
do so, we consider only the one-dimensional case. Here we focus on the birth-death 
processes since the one-dimensional diffusion processes are in parallel. The crite- 
rion for strong ergodicity was obtained recently by Zhang, Lin and Hou (2000) , and 
extended by Zhang (2001), using a different approach, to a larger class of Markov 
chains. The criteria for logarithmic Sobolev, Nash inequalities, and the discrete 
spectrum (no continuous spectrum and all eigenvalues have finite multiplicity) were 
obtained by Bobkov and Gotze (1999) and Mao (2000, 2002a, b), respectively, based 
on the weighted Hardy inequality (see also Miclo (1999), Wang (2000), Gong and 
Wang (2002)). It is understood now the results can also be deduced from gener- 
alizations of the variational formulas discussed in this paper (cf. Chen (2001b)). 
Finally, we summarize these results in Theorem 4.3 and Table 4.2. The table is 
arranged in such an order that the property in the latter line is stronger than the 
property in the former line. The only exception is that even though the strong 
ergodicity is often stronger than the logarithmic Sobolev inequality, they are not 
comparable in general, as mentioned in Part M. 

Theorem 4.3 (Chen (2001a)). For birth-death processes with birth rates bi{i ^ 0) 
and death rates ai{i ^ 1), ten criteria are listed in Table 4-2. Recall the sequence {fXi) 
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defined in Part II and set /u[i, k] = X^^<j<fe fJ-j- The notion "(*) & • • • " appeared in 
Table 4-2 means that one requires the uniqueness condition in the first line plus the 
condition "■ • ■ ". The notion "(e) " in the last line means that there is still a small 
room (1 < g < 2) left from completeness. 



Property 


Criterion 


Uniqueness 


-^j-fi[0,n\ = oo (*) 


R.ccurrencc 


— T- = °° 


Ergodicity 


(*) & /i[0, oo) < oo 


Exponential ergodicity 
L^-exp. convergence 


(*) & sup/u[n, oo) — — < oo 

j^n— 1 ■' ■' 


Discrete spectrum 


(*) & Hm sup /Lt[A;,oo) — ^ = 


Log. Sobolev inequality 


(*) & sup^[n, oo)log[/i[n, oo)~"^] — ^<oo 

n>l j^„-l^J^J 


Strong ergodicity 
L^-exp. convergence 




Nash inequality 


(*) & sup^[n,oo)(«-2)/(«-i) ^ \ <oo (e) 



Table 4.2 Ten criteria for birth-death processes 
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